Light-Field Imaging 101

A comprehensive introduction to unfocused light-field imaging (plenoptic 1.0)

Intended Audience

This guide aims to expand upon existing unfocused light-field (plenoptic 1.0) imaging learning material by providing detailed Python implementations of light-field image processing along with examples. The light-field image processing workflow closely follows that of the MATLAB-based Light-Field Imaging Toolkit (Bolan et al. 2016), but leverages Python’s open-source ecosystem and Quarto’s enhanced capabilities for visualization and LaTeX formula integration.

By focusing specifically on the image processing aspects, this guide helps readers better understand the technical limitations and practical considerations of light-field imaging techniques. This guide assumes the reader is familiar with unfocused light-field imaging. Such an understanding can be achieved by browsing plenoptic.info or reading the unfocused light-field part of the Light-Field Camera Working Principles chapter (Pages 11-25) of the book Development and Application of Light-Field Cameras in Fluid Measurements (Shi and New 2023).

Raw images

To illustrate light-field image processing concepts, a synthetic unfocused light-field image was generated and is shown in Figure 1. The image describes the raw pixel intensities as recorded by an unfocused light-field camera. Every lenslet of the camera projects the incident radiance into a defocused intensity distribution pattern. This manifests as a spatially constrained blur where the light energy from each microlens is distributed across multiple sensor pixels rather than being concentrated at the expected conjugate position. The image was generated using a commercial ray tracing software (OpticStudio, ANSYS).

Figure 1: Synthetic unfocused light-field image. The hexagonal patterning in the image is characteristic of the projection process by the microlens array.

Calibration

The calibration process seeks to pinpoint the centroid of each lenslet’s corresponding sensor region. A calibration image is acquired with the main lens’s aperture reduced to a minimum. The resulting calibration image consist of an array of small bright spots as shown in Figure 2.

Figure 2: Synthetic light-field calibration image.

References

Bolan, Jeffrey, Elise Hall, Chris Clifford, and Brian Thurow. 2016. “Light-Field Imaging Toolkit.” SoftwareX 5: 101–6.

Shi, Shengxian, and TH New. 2023. Development and Application of Light-Field Cameras in Fluid Measurements. Springer.